Replication studies considered harmful
نویسنده
چکیده
Context: There is growing interest in establishing software engineering as an evidence-based discipline. To that end, replication is often used to gain confidence in empirical findings, as opposed to reproduction where the goal is showing the correctness, or validity of the published results. Objective: To consider what is required for a replication study to confirm the original experiment and apply this understanding in software engineering. Method: Simulation is used to demonstrate why the prediction interval for confirmation can be surprisingly wide. This analysis is applied to three recent replications. Results: It is shown that because the prediction intervals are wide, almost all replications are confirmatory, so in that sense there is no ‘replication crisis’, however, the contributions to knowledge are negligible. Conclusion: Replicating empirical software engineering experiments, particularly if they are under-powered or under-reported, is a waste of scientific resources. By contrast, meta-analysis is strongly advocated so that all relevant experiments are combined to estimate the population effect.
منابع مشابه
Microsatellite (SSR) amplification by PCR usually led to polymorphic bands: Evidence which shows replication slippage occurs in extend or nascent DNA strands
Microsatellites or simple sequence repeats (SSRs) are very effective molecular markers in population genetics, genome mapping, taxonomic study and other large-scale studies. Variation in number of tandem repeats within microsatellite refers to simple sequence length polymorphism (SSLP); but there are a few studies that are showed SSRs replication slippage may be occurred during in vitro amplifi...
متن کاملStudies on the replication of African Horse Sickness Virus in two different cell line cultures
متن کامل
Parasites Sustain and Enhance RNA-Like Replicators through Spatial Self-Organisation
In a prebiotic RNA world, parasitic behaviour may be favoured because template dependent replication happens in trans, thus being altruistic. Spatially extended systems are known to reduce harmful effects of parasites. Here we present a spatial system to show that evolution of replication is (indirectly) enhanced by strong parasites, and we characterise the phase transition that leads to this m...
متن کاملEffects of Sodium Valproate on the Replication of Herpes Simplex Virus Type 1: An in Vitro Study
Background: Sodium valproate, an anticonvulsant drug, is reported to stimulate Human Immunodeficiency Virus type 1 and Human cytomegalovirus replication. Since epileptic patients undergoing sodium valproate therapy may suffer from various virus infections, the effect of this drug on replication of viruses especially those affecting neuronal tissues such as Herpes simplex virus type 1 is worthy ...
متن کاملEffect of Activation and Inhibition of Cellular PKR on Coxsackievirus B3 Replication
The ds-RNA activated protein kinase (PKR) is a serine-threonine kinase with MW of 68 KDa. It belongs to a family of kinases that control one of the translational initiation factors, eIF2. PKR is produced at high level in response to viral infection. This protein by phosphorylating eIF2 inhibits cellular protein synthesis. In this study, the effect of gamma interferon (IFN-γ), an activator, and ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1802.04580 شماره
صفحات -
تاریخ انتشار 2018